In Exploratory Data Analysis of tabular data, univariate analysis is the first step. It consists in exploring, summarizing, visualizing columns of a dataset. In this workbook we focus on univariate numerical samples. We explore techniques for:
This is also an opportunity to:
gssr and gssrdoc
If the required packages have not (yet) been installed, install them.
stopifnot(
require(skimr), # Univariate summaries from the shelf
require(lobstr), # R introspection
require(rlang), # R introspection
require(glue), # Like formatted strings
require(gssr),
require(gssrdoc),
require(fs), # File manipulation
require(patchwork), # piecing ggplots together
require(tidyverse) # What else?
)Load gss_sub which is much smaller than gss_all. Have a glimpse.
gss_sub?In the sequel, we explore the age distribution as is the age column was a genuine univariate sample. This is done for teaching purpose. The age column is not collected by repeatedy picking individuals uniformly at random from a fixed population.
Indeed the age column is a union of samples collected every year or every two years since 1972. The American population has changed thoughout the five decades.
Moreover, yearly samples are not i.i.d. samples from the whole population. The sampling methods have varied over time. Sampling methods rely on multistage stratified sampling and quotas.
age columnFor column age, disregarding any weighting process
age.age distribution according to sex.varwidth=T?notch=T?stat_boxplot() and geom_boxplot()?age distributionsex
age distribution histograms for each sex on the same plotsex and year
frame is determined by year
Histograms are used to sketch possibly (absolutely) continuous distributions by using piecewise constant approximations of density functions. Histograms can also be viewed as column plots for binned data (that is discretizations of “continuous” data).
age data
[18-25[, [25, 35[, [35,50[, [50, 65[, [65,+∞[
age according to defined breaks using cut()
geom_bar() or geom_col()
Demographers use population pyramids to sketch the age distribution in a population. Population pyramids are special facetted histograms or barplots.
gss sample.year
Histograms deliver piecewise constant estimations/approximations of a population density. If we suspect the population density to be smooth, it is sensible to try to build smooth estimates/aproximations of the population density. This is the purpose of density estimates.
sex
sex and year
geom_histogram() use aes(y=after_stat(density)))Use stat_density()
Build violine plots for age distribution (use geom_violine()).
Not all probability distributions have densities, but all are characterized by their Cumulative Distribution Functions (CDFs). Each sample defines an Empirical Cumulative Distribution Function (ECDF).
age ECDF using stat_ecdf()
sex, then by year and sex
R ecdf() and stat_function() to draw the same plot.age distributions for women and men using the Kolmogorov-Smirnov statistic (ks.test())The quantile function of a probability distribution is the (generalized, left-continuous) inverse of its CDF. Quantile functions are useful devices in EDA and random generation.
age empirical distributionage empirical distributions for men and womenecdf() does)age distribution for women and men with base R qqplot()
age distribution for women and men using ggplot2.R lazy loading?skimrgssrgssrdoc